Name: Loubna Sasso - nyc SQF

Originality declaration

I, \[**Loubna Sasso**\], confirm that the work presented in this assessment is my own. Where information has been derived from other sources, I confirm that this has been indicated in the work.

date: 02 February, 2022

Start your response here

01 Initial project scope

SQF policy has been the subject of racial profiling over the years, therefore analyzing where SQF polcies are being implemented across different areas of NYC and who they are targeted towards is essential in order to help overcome racial profiling by informing policies for specific precincts to ensure equality. Moreover Research shows that “persons of African and Hispanic descent were stopped more frequently than white people, even after controlling for precinct variability and race-specific estimates of crime participation.” Therefore analyzing where these differences occur in different parts on NYC can be insightful to implement necessary changes.

The databasets used for this analysis comes from NYC open data and New York City Police Department. The New York City Police Department Dataset contains 9545 SQF entries contains entries for and 83 different fields. The dataset includes important information on the various SQF throughout 2020, at different months, and time of day as well as the race and gender of the people being questioned.

The fields that will be used for this analysis include the latitude and longitude coordinates, suspect race description and, month. I will analyse the pattern of sqf on black people over the summer period (june,july,august,spetember). This process will repeated for ‘white people’ if i have time during the 6 hours given, and a comparison between sqf of black vs white people will be made

Research Question:

What patterns off risk (SQF) can be observed in the different precincts in NYC for black people and how might this pattern change in other racial groups?

Hypothesis:

HYPOTHESIS 1 h0: There are no noticeable spacial cluster pattern of SQF policy that can be observed for the black people across NYC h1: There are noticeable spacial cluster pattern of SQF policy that can be observed for the black people across NYC

HYPOTHESIS 2 (moran, getis gerrys) Ho: There are no similarities in spatial autocorrelation of (SQF) policy between black and white people
H1: There are similarities in spatial autocorrelation of (SQF) policy between black and white people

Data Limitations

• Point data also lie outside the boundaries of nyc boundaries that could influence what’s happening inside
• Is it ethical to include peoples names in the data provided?

Analysis Assumptions and focus areas

• Assume that points outside NYC boundary have no influence on the outcome of data inside NYC
• Focus on black community and then possibly white community if i have time
• Assume data is complete regardless of na values

Logical Plan

Preprocess the data: - load in data, filter data to black people being questioned and transform it - transform lat and long values to point data, make all points distinct, clip all point to NYC boundary - st_join points to spatial data: group by, summarize and get a density map

Analyzing the data: - Descriptive statistics of all SQF on black people density data: BOX PLOT - Explore clustering of SQF using Point pattern analysis: kerney, ripleys and DBSCAN - Explore different Global methods to analyze levels Spatial autocorrelation of SQF in NYC - Explore different local methods to analyze clustering across communities in NYC

02 Load in libraries

library(spatstat)

## Loading required package: spatstat.data

## Loading required package: spatstat.geom

## spatstat.geom 2.3-0

## Loading required package: spatstat.core

## Loading required package: nlme

## Loading required package: rpart

## spatstat.core 2.3-1

## Loading required package: spatstat.linnet

## spatstat.linnet 2.3-0

## 
## spatstat 2.2-0       (nickname: 'That's not important right now') 
## For an introduction to spatstat, type 'beginner'

library(here)

## here() starts at C:/Users/loubn/Desktop/old laptop uni/CASA-GIS/GIS FINAL PREP/casa0005-final-exam-LoubnaSasso

library(sp)
library(rgeos)

## rgeos version: 0.5-8, (SVN revision 679)
##  GEOS runtime version: 3.9.1-CAPI-1.14.2 
##  Please note that rgeos will be retired by the end of 2023,
## plan transition to sf functions using GEOS at your earliest convenience.
##  GEOS using OverlayNG
##  Linking to sp version: 1.4-6 
##  Polygon checking: TRUE

library(maptools)

## Checking rgeos availability: TRUE
## Please note that 'maptools' will be retired by the end of 2023,
## plan transition at your earliest convenience;
## some functionality will be moved to 'sp'.

library(GISTools)

## Loading required package: RColorBrewer

## Loading required package: MASS

## 
## Attaching package: 'MASS'

## The following object is masked from 'package:spatstat.geom':
## 
##     area

library(tmap)
library(sf)

## Linking to GEOS 3.9.1, GDAL 3.2.1, PROJ 7.2.1

library(geojson)

## 
## Attaching package: 'geojson'

## The following object is masked from 'package:graphics':
## 
##     polygon

library(geojsonio)

## Registered S3 method overwritten by 'geojsonsf':
##   method        from   
##   print.geojson geojson

## 
## Attaching package: 'geojsonio'

## The following object is masked from 'package:base':
## 
##     pretty

library(tmaptools)
library(stringr)
library(tidyverse)

## Registered S3 method overwritten by 'cli':
##   method     from         
##   print.boxx spatstat.geom

## -- Attaching packages --------------------------------------- tidyverse 1.3.1 --

## v ggplot2 3.3.5     v purrr   0.3.4
## v tibble  3.1.6     v dplyr   1.0.7
## v tidyr   1.1.4     v forcats 0.5.1
## v readr   2.1.0

## -- Conflicts ------------------------------------------ tidyverse_conflicts() --
## x dplyr::collapse() masks nlme::collapse()
## x dplyr::filter()   masks stats::filter()
## x dplyr::lag()      masks stats::lag()
## x dplyr::select()   masks MASS::select()

library(raster)

## 
## Attaching package: 'raster'

## The following object is masked from 'package:dplyr':
## 
##     select

## The following object is masked from 'package:MASS':
## 
##     select

## The following object is masked from 'package:nlme':
## 
##     getData

library(fpc)
library(dbscan)

## 
## Attaching package: 'dbscan'

## The following object is masked from 'package:fpc':
## 
##     dbscan

library(ggplot2)
library(OpenStreetMap)
library(janitor)

## 
## Attaching package: 'janitor'

## The following object is masked from 'package:raster':
## 
##     crosstab

## The following objects are masked from 'package:stats':
## 
##     chisq.test, fisher.test

library(dplyr)
library(spdep)

## Loading required package: spData

## To access larger datasets in this package, install the spDataLarge
## package with: `install.packages('spDataLarge',
## repos='https://nowosad.github.io/drat/', type='source')`

library(plotly)

## 
## Attaching package: 'plotly'

## The following object is masked from 'package:raster':
## 
##     select

## The following object is masked from 'package:ggplot2':
## 
##     last_plot

## The following object is masked from 'package:MASS':
## 
##     select

## The following object is masked from 'package:stats':
## 
##     filter

## The following object is masked from 'package:graphics':
## 
##     layout

library(rgdal)

## Please note that rgdal will be retired by the end of 2023,
## plan transition to sf/stars/terra functions using GDAL and PROJ
## at your earliest convenience.
## 
## rgdal: version: 1.5-27, (SVN revision 1148)
## Geospatial Data Abstraction Library extensions to R successfully loaded
## Loaded GDAL runtime: GDAL 3.2.1, released 2020/12/29
## Path to GDAL shared files: C:/Users/loubn/Documents/R/win-library/4.1/rgdal/gdal
## GDAL binary built with GEOS: TRUE 
## Loaded PROJ runtime: Rel. 7.2.1, January 1st, 2021, [PJ_VERSION: 721]
## Path to PROJ shared files: C:/Users/loubn/Documents/R/win-library/4.1/rgdal/proj
## PROJ CDN enabled: FALSE
## Linking to sp version:1.4-6
## To mute warnings of possible GDAL/OSR exportToProj4() degradation,
## use options("rgdal_show_exportToProj4_warnings"="none") before loading sp or rgdal.
## Overwritten PROJ_LIB was C:/Users/loubn/Documents/R/win-library/4.1/rgdal/proj

library(broom)
library(mapview)
library(crosstalk)
library(car)

## Loading required package: carData

## 
## Attaching package: 'car'

## The following object is masked from 'package:dplyr':
## 
##     recode

## The following object is masked from 'package:purrr':
## 
##     some

## The following object is masked from 'package:spatstat.core':
## 
##     bc

## The following object is masked from 'package:spatstat.geom':
## 
##     ellipse

library(fs)
library(tidypredict)
library(corrr)

## 
## Attaching package: 'corrr'

## The following object is masked from 'package:raster':
## 
##     stretch

library(spatialreg)

## Loading required package: Matrix

## 
## Attaching package: 'Matrix'

## The following objects are masked from 'package:tidyr':
## 
##     expand, pack, unpack

## 
## Attaching package: 'spatialreg'

## The following objects are masked from 'package:spdep':
## 
##     aple, aple.mc, aple.plot, as.spam.listw, as_dgRMatrix_listw,
##     as_dsCMatrix_I, as_dsCMatrix_IrW, as_dsTMatrix_listw,
##     can.be.simmed, cheb_setup, create_WX, do_ldet, eigen_pre_setup,
##     eigen_setup, eigenw, errorsarlm, get.ClusterOption,
##     get.coresOption, get.mcOption, get.VerboseOption,
##     get.ZeroPolicyOption, GMargminImage, GMerrorsar, griffith_sone,
##     gstsls, Hausman.test, impacts, intImpacts, invIrM, invIrW,
##     Jacobian_W, jacobianSetup, l_max, lagmess, lagsarlm, lextrB,
##     lextrS, lextrW, lmSLX, localAple, LU_prepermutate_setup, LU_setup,
##     Matrix_J_setup, Matrix_setup, mcdet_setup, MCMCsamp, ME, mom_calc,
##     mom_calc_int2, moments_setup, powerWeights, sacsarlm,
##     SE_classic_setup, SE_interp_setup, SE_whichMin_setup,
##     set.ClusterOption, set.coresOption, set.mcOption,
##     set.VerboseOption, set.ZeroPolicyOption, similar.listw, spam_setup,
##     spam_update_setup, SpatialFiltering, spautolm, spBreg_err,
##     spBreg_lag, spBreg_sac, stsls, subgraph_eigenw, trW

library(spdep)
library(spgwr)

## NOTE: This package does not constitute approval of GWR
## as a method of spatial analysis; see example(gwr)

library(joineR)

## Loading required package: survival

library(units)

## udunits database from C:/Users/loubn/Documents/R/win-library/4.1/units/share/udunits/udunits2.xml

library(ggplot2)
library(xlsx)
library(readxl)

03 Load in data +transform Coordinates

#read in Police data and transform it
police <- st_read(here::here ("Data", "Police Precincts.geojson")) %>%
  st_transform(., 32618) #2908

## Reading layer `Police Precincts' from data source 
##   `C:\Users\loubn\Desktop\old laptop uni\CASA-GIS\GIS FINAL PREP\casa0005-final-exam-LoubnaSasso\Data\Police Precincts.geojson' 
##   using driver `GeoJSON'
## Simple feature collection with 77 features and 3 fields
## Geometry type: MULTIPOLYGON
## Dimension:     XY
## Bounding box:  xmin: -74.25559 ymin: 40.49613 xmax: -73.70001 ymax: 40.91553
## Geodetic CRS:  WGS 84

#read in sqf data, remove all na values and clean names
sqf  <- read_excel(here::here ("Data", "sqf-2020.xlsx"))%>%
  clean_names()

04 Initianl Data Cleaning and wrangling

for sql: -select fields: suspect_race_description, stop_location_x, stop_location_x, suspect_arrest_offense, month2 -filter: Black, white, white hispanic -drop n/a values -transform

#1) for sqf csv data i want to:

#select columns you want. 
sqf <-  sqf%>%
  dplyr::select(suspect_race_description, stop_location_x, stop_location_y, month2)


# first filter only black and drop na vals in lat and long
sqf_black <-  sqf%>%
  filter(suspect_race_description == "BLACK")%>%
  filter(month2 == "June" | month2 == "July" | month2 == "August" | month2 == "September" )%>%
  drop_na(stop_location_x)%>%
  drop_na(stop_location_y)
                  
  
# first filter only white and drop any na vals in lat and long. this will be used later to compare if i have time
sqf_white <-  sqf%>%
  filter(suspect_race_description == "WHITE HISPANIC" | suspect_race_description == "WHITE")%>%
  filter(month2 == "June" | month2 == "July" | month2 == "August" | month2 == "September" )%>%
  drop_na(stop_location_x)%>%
  drop_na(stop_location_y)

  

#2) transform csv sqf_black to point data: st_as sf. 
sqf_black_sf <- sqf_black%>%
  st_as_sf(.,coords = c ( "stop_location_x", "stop_location_y"), crs=2908)%>%
  st_transform(.,32618)

  
#make all points distinct 
sqf_black_sf <- sqf_black_sf%>%
  distinct(geometry, .keep_all=T)


#3) overlay point data to police districts and clip if required
tmap_mode("view")

## tmap mode set to interactive viewing

tm_shape(police) +
  tm_polygons(col = NA, alpha = 0.5) +
tm_shape(sqf_black_sf) +
  tm_dots(col = "blue")

#clipping point to xxx map boundary
sqf_black_sf <- sqf_black_sf [police,]

#plot again to see if they are clipped
tmap_mode("view")

## tmap mode set to interactive viewing

tm_shape(police) +
  tm_polygons(col = NA, alpha = 0.5) +
tm_shape(sqf_black_sf) +
  tm_dots(col = "blue")

05 Density map and Descriptive Statistics: sqf black

lets get an idea of our sqf black data density and distribution

#1) observe areas of highest and lowest density of sqf-black
#could similar  densities in adjacent districts mean there is some degree of clustering happening in these areas?


#st_join point data to the police map to allow for continuous data observation
police_sqf_black <- police%>%
  st_join(sqf_black_sf)

#count no. of points in each precinct and add area and density 
police_sqf_black <- police_sqf_black%>%
  add_count(precinct)%>% #counts the no. of points in each precinct (n)
  janitor::clean_names()%>%
  mutate(area = st_area(.))%>% #get area
  mutate (density = n/area)%>%  #get density
  dplyr::select(density, area, precinct, n) #narror down the colomns you want


#group by borough
police_sqf_black <-  police_sqf_black%>%
  group_by(precinct)%>%
  summarise(density = first(density), precinct = first(precinct), area = first(area), 
            count = first(n))



#quick choropleth map based on DENSITY  
tm_shape(police_sqf_black) +
    tm_polygons("density",
        style="jenks",
        palette="PuOr",
        midpoint=NA,
        popup.vars=c("precinct", "density"),
        title="SQF-Black people Density")

#2) Box plot

#first make sure density col is numeric
police_sqf_black$density_numeric <- as.numeric(police_sqf_black$density)

#boxplot(police_sqf_black$density_numeric)
boxplot(police_sqf_black$density_numeric,
main = "SQF-Black people Density Distribution ",
xlab = "density ",
ylab = "",
col = "orange",
border = "brown",
horizontal = TRUE
)

Highest levels of sqf amongst black people are indicated in dark purple in precincts across Manhattan, west of Bronx and north of Brooklyn. it can also be seen that precincts closer together follow more or less similar sqf of black people density which could indicate similarity in governance in these precincts that are influenced by neighbouring precincts, therefore spatial autocorrelation and clustering occurance.

We can also see the density distribution is positively skewed, meaning that most of the precints in Nyc exhibit lower densities in of sqf amongst black people, however there are still a few areas with high densities. Lets explore further the spatial patterns of these density distribution across the different precinct through point pattern analysis and later morans I

##06 Cluster Analysis: Point pattern Analysis:

ANalysis process: 1. create sp and sf objects 2. Density kernel (which confirms the density map created above) 3. Ripley K (over quadrat) analysis to test clustering present 4. DBSCAN to show where clusters are happening

1.Create sp and ppp objects from the sf point data in order to proceed with ripley k and DBSCAN

#1) now set a window as the precinct nyc boundary 
window <- as.owin(police)
plot(window)

#2A)  Create a sp object for eviction (sf to sp)
sqf_black_sp <- sqf_black_sf %>%
  as(., 'Spatial')

#2B) Create a ppp object
sqf_black_ppp <- ppp(x=sqf_black_sp@coords[,1],
                          y=sqf_black_sp@coords[,2],
                          window=window)

#sqf_black_sp@coords[,1]



#3) have a look at the ppp object

sqf_black_ppp %>%
  plot(.,pch=16,cex=0.5, 
       main="SQF Black people")

kernel Density:

This confirms the density map produced earlier.

#uses ppp data 
sqf_black_ppp %>%
  density(., sigma=500) %>%
  plot()

We can see from this map that the that the precincts that fall in manhattan , parts of Bronx shows highest densities of SQF concentration, similar to the first density diagram produced. Now lets explore further if the points exhibit clustering using ripleys k.

Ripleys k

“To determining whether a pattern exists in the data is to compare the observed distribution of points to an idealised random distribution. If the points do not conform to a random distribution, then it is likely that there is some unobserved factor or factors which could be causing non-random clustering or dispersal to occur” (Adam Dennett and Sam).

Quadrat analysis does not account for different spatial unit sizes such the different sized precincts in NYC, therefore to overcome this limitation, i will jump straight in to a ripleys k test to check if the SQF cases of black people exhibits clustering, This is done by comparing the observed distribution of points with the Poisson random model for a whole range of different distance radii.

#using the ppp data, plot the ripleys K
Ripleys_K <- sqf_black_ppp%>%
  Kest(., correction="border") %>%
  plot()

Kval_global <- as.data.frame(Kest(sqf_black_ppp, correction = "border"))
Kval_global

##               r         theo       border
## 1       0.00000 0.000000e+00         0.00
## 2      22.79911 1.632998e+03     11901.04
## 3      45.59822 6.531993e+03     54863.40
## 4      68.39734 1.469699e+04     93369.07
## 5      91.19645 2.612797e+04    156446.40
## 6     113.99556 4.082496e+04    228775.26
## 7     136.79467 5.878794e+04    328605.76
## 8     159.59379 8.001692e+04    447167.33
## 9     182.39290 1.045119e+05    560608.39
## 10    205.19201 1.322729e+05    666790.28
## 11    227.99112 1.632998e+05    822915.30
## 12    250.79023 1.975928e+05    967560.83
## 13    273.58935 2.351518e+05   1130273.23
## 14    296.38846 2.759767e+05   1336499.26
## 15    319.18757 3.200677e+05   1509229.26
## 16    341.98668 3.674246e+05   1683968.37
## 17    364.78580 4.180476e+05   1891542.38
## 18    387.58491 4.719365e+05   2116443.25
## 19    410.38402 5.290915e+05   2371491.31
## 20    433.18313 5.895124e+05   2632645.29
## 21    455.98224 6.531993e+05   2862262.06
## 22    478.78136 7.201523e+05   3107248.72
## 23    501.58047 7.903712e+05   3310106.09
## 24    524.37958 8.638561e+05   3625339.91
## 25    547.17869 9.406070e+05   3884404.02
## 26    569.97780 1.020624e+06   4267395.80
## 27    592.77692 1.103907e+06   4551358.65
## 28    615.57603 1.190456e+06   4921733.80
## 29    638.37514 1.280271e+06   5231634.26
## 30    661.17425 1.373352e+06   5552652.60
## 31    683.97337 1.469699e+06   5895016.32
## 32    706.77248 1.569311e+06   6227699.18
## 33    729.57159 1.672190e+06   6614419.70
## 34    752.37070 1.778335e+06   6977882.03
## 35    775.16981 1.887746e+06   7264778.61
## 36    797.96893 2.000423e+06   7575702.49
## 37    820.76804 2.116366e+06   7935309.18
## 38    843.56715 2.235575e+06   8316194.52
## 39    866.36626 2.358050e+06   8677560.44
## 40    889.16538 2.483790e+06   9095402.71
## 41    911.96449 2.612797e+06   9511941.25
## 42    934.76360 2.745070e+06   9997693.18
## 43    957.56271 2.880609e+06  10363077.15
## 44    980.36182 3.019414e+06  10861423.31
## 45   1003.16094 3.161485e+06  11372886.99
## 46   1025.96005 3.306822e+06  11704053.34
## 47   1048.75916 3.455424e+06  12302945.07
## 48   1071.55827 3.607293e+06  12874746.77
## 49   1094.35739 3.762428e+06  13291070.80
## 50   1117.15650 3.920829e+06  13750427.22
## 51   1139.95561 4.082496e+06  14236622.60
## 52   1162.75472 4.247429e+06  14649409.33
## 53   1185.55383 4.415628e+06  15070560.45
## 54   1208.35295 4.587092e+06  15574522.07
## 55   1231.15206 4.761823e+06  16114500.91
## 56   1253.95117 4.939820e+06  16491338.94
## 57   1276.75028 5.121083e+06  16884198.59
## 58   1299.54940 5.305612e+06  17346373.34
## 59   1322.34851 5.493406e+06  18054670.59
## 60   1345.14762 5.684467e+06  18682228.85
## 61   1367.94673 5.878794e+06  19417468.72
## 62   1390.74584 6.076387e+06  19991285.51
## 63   1413.54496 6.277246e+06  20505256.11
## 64   1436.34407 6.481370e+06  20858307.53
## 65   1459.14318 6.688761e+06  21690633.95
## 66   1481.94229 6.899418e+06  22543926.56
## 67   1504.74140 7.113341e+06  22932505.59
## 68   1527.54052 7.330530e+06  23937425.05
## 69   1550.33963 7.550984e+06  24784454.00
## 70   1573.13874 7.774705e+06  25466470.51
## 71   1595.93785 8.001692e+06  26453468.20
## 72   1618.73697 8.231945e+06  26809224.38
## 73   1641.53608 8.465463e+06  27512515.29
## 74   1664.33519 8.702248e+06  28624783.48
## 75   1687.13430 8.942299e+06  29119030.14
## 76   1709.93341 9.185616e+06  30126251.23
## 77   1732.73253 9.432198e+06  31056845.79
## 78   1755.53164 9.682047e+06  31790604.24
## 79   1778.33075 9.935162e+06  32569564.17
## 80   1801.12986 1.019154e+07  33403977.91
## 81   1823.92898 1.045119e+07  33828068.89
## 82   1846.72809 1.071410e+07  34607635.48
## 83   1869.52720 1.098028e+07  36294671.33
## 84   1892.32631 1.124973e+07  37191279.34
## 85   1915.12542 1.152244e+07  37939674.77
## 86   1937.92454 1.179841e+07  39174179.11
## 87   1960.72365 1.207766e+07  39849630.24
## 88   1983.52276 1.236016e+07  41004097.35
## 89   2006.32187 1.264594e+07  42151433.69
## 90   2029.12099 1.293498e+07  42820802.52
## 91   2051.92010 1.322729e+07  43693489.00
## 92   2074.71921 1.352286e+07  45434396.65
## 93   2097.51832 1.382170e+07  46057820.81
## 94   2120.31743 1.412380e+07  47466476.98
## 95   2143.11655 1.442917e+07  49506160.26
## 96   2165.91566 1.473781e+07  50369084.22
## 97   2188.71477 1.504971e+07  51424931.00
## 98   2211.51388 1.536488e+07  54062187.64
## 99   2234.31299 1.568332e+07  55736129.46
## 100  2257.11211 1.600502e+07  58072904.50
## 101  2279.91122 1.632998e+07  60650153.12
## 102  2302.71033 1.665822e+07  62315730.92
## 103  2325.50944 1.698971e+07  63759731.22
## 104  2348.30856 1.732448e+07  64776632.83
## 105  2371.10767 1.766251e+07  67522267.19
## 106  2393.90678 1.800381e+07  68868785.19
## 107  2416.70589 1.834837e+07  69661526.88
## 108  2439.50500 1.869620e+07  70494748.83
## 109  2462.30412 1.904729e+07  72233911.34
## 110  2485.10323 1.940165e+07  73081329.35
## 111  2507.90234 1.975928e+07  74403301.45
## 112  2530.70145 2.012017e+07  75521893.22
## 113  2553.50057 2.048433e+07  76329519.43
## 114  2576.29968 2.085176e+07  77708968.58
## 115  2599.09879 2.122245e+07  78805453.80
## 116  2621.89790 2.159640e+07  79577173.60
## 117  2644.69701 2.197363e+07  80009819.01
## 118  2667.49613 2.235411e+07  81352129.14
## 119  2690.29524 2.273787e+07  81713694.16
## 120  2713.09435 2.312489e+07  81860579.95
## 121  2735.89346 2.351518e+07  82049433.11
## 122  2758.69258 2.390873e+07  84355900.07
## 123  2781.49169 2.430555e+07  85670049.85
## 124  2804.29080 2.470563e+07  86796463.94
## 125  2827.08991 2.510898e+07  86233256.89
## 126  2849.88902 2.551560e+07  85907848.38
## 127  2872.68814 2.592548e+07  87697595.22
## 128  2895.48725 2.633863e+07  88493038.26
## 129  2918.28636 2.675504e+07  90300863.35
## 130  2941.08547 2.717473e+07  90300863.35
## 131  2963.88459 2.759767e+07  91811688.61
## 132  2986.68370 2.802388e+07 104944246.60
## 133  3009.48281 2.845336e+07 106300115.42
## 134  3032.28192 2.888611e+07 108062744.88
## 135  3055.08103 2.932212e+07 109337261.57
## 136  3077.88015 2.976139e+07 108740679.29
## 137  3100.67926 3.020394e+07 110638895.64
## 138  3123.47837 3.064975e+07          NaN
## 139  3146.27748 3.109882e+07          NaN
## 140  3169.07659 3.155116e+07          NaN
## 141  3191.87571 3.200677e+07          NaN
## 142  3214.67482 3.246564e+07          NaN
## 143  3237.47393 3.292778e+07          NaN
## 144  3260.27304 3.339318e+07          NaN
## 145  3283.07216 3.386185e+07          NaN
## 146  3305.87127 3.433379e+07          NaN
## 147  3328.67038 3.480899e+07          NaN
## 148  3351.46949 3.528746e+07          NaN
## 149  3374.26860 3.576920e+07          NaN
## 150  3397.06772 3.625420e+07          NaN
## 151  3419.86683 3.674246e+07          NaN
## 152  3442.66594 3.723400e+07          NaN
## 153  3465.46505 3.772879e+07          NaN
## 154  3488.26417 3.822686e+07          NaN
## 155  3511.06328 3.872819e+07          NaN
## 156  3533.86239 3.923279e+07          NaN
## 157  3556.66150 3.974065e+07          NaN
## 158  3579.46061 4.025178e+07          NaN
## 159  3602.25973 4.076617e+07          NaN
## 160  3625.05884 4.128383e+07          NaN
## 161  3647.85795 4.180476e+07          NaN
## 162  3670.65706 4.232895e+07          NaN
## 163  3693.45618 4.285641e+07          NaN
## 164  3716.25529 4.338713e+07          NaN
## 165  3739.05440 4.392112e+07          NaN
## 166  3761.85351 4.445838e+07          NaN
## 167  3784.65262 4.499890e+07          NaN
## 168  3807.45174 4.554269e+07          NaN
## 169  3830.25085 4.608975e+07          NaN
## 170  3853.04996 4.664007e+07          NaN
## 171  3875.84907 4.719365e+07          NaN
## 172  3898.64819 4.775050e+07          NaN
## 173  3921.44730 4.831062e+07          NaN
## 174  3944.24641 4.887401e+07          NaN
## 175  3967.04552 4.944066e+07          NaN
## 176  3989.84463 5.001057e+07          NaN
## 177  4012.64375 5.058376e+07          NaN
## 178  4035.44286 5.116021e+07          NaN
## 179  4058.24197 5.173992e+07          NaN
## 180  4081.04108 5.232290e+07          NaN
## 181  4103.84019 5.290915e+07          NaN
## 182  4126.63931 5.349866e+07          NaN
## 183  4149.43842 5.409144e+07          NaN
## 184  4172.23753 5.468748e+07          NaN
## 185  4195.03664 5.528679e+07          NaN
## 186  4217.83576 5.588937e+07          NaN
## 187  4240.63487 5.649521e+07          NaN
## 188  4263.43398 5.710432e+07          NaN
## 189  4286.23309 5.771669e+07          NaN
## 190  4309.03220 5.833233e+07          NaN
## 191  4331.83132 5.895124e+07          NaN
## 192  4354.63043 5.957341e+07          NaN
## 193  4377.42954 6.019885e+07          NaN
## 194  4400.22865 6.082756e+07          NaN
## 195  4423.02777 6.145953e+07          NaN
## 196  4445.82688 6.209476e+07          NaN
## 197  4468.62599 6.273326e+07          NaN
## 198  4491.42510 6.337503e+07          NaN
## 199  4514.22421 6.402007e+07          NaN
## 200  4537.02333 6.466837e+07          NaN
## 201  4559.82244 6.531993e+07          NaN
## 202  4582.62155 6.597477e+07          NaN
## 203  4605.42066 6.663286e+07          NaN
## 204  4628.21978 6.729423e+07          NaN
## 205  4651.01889 6.795886e+07          NaN
## 206  4673.81800 6.862676e+07          NaN
## 207  4696.61711 6.929792e+07          NaN
## 208  4719.41622 6.997235e+07          NaN
## 209  4742.21534 7.065004e+07          NaN
## 210  4765.01445 7.133100e+07          NaN
## 211  4787.81356 7.201523e+07          NaN
## 212  4810.61267 7.270272e+07          NaN
## 213  4833.41179 7.339348e+07          NaN
## 214  4856.21090 7.408750e+07          NaN
## 215  4879.01001 7.478479e+07          NaN
## 216  4901.80912 7.548535e+07          NaN
## 217  4924.60823 7.618917e+07          NaN
## 218  4947.40735 7.689626e+07          NaN
## 219  4970.20646 7.760661e+07          NaN
## 220  4993.00557 7.832023e+07          NaN
## 221  5015.80468 7.903712e+07          NaN
## 222  5038.60379 7.975727e+07          NaN
## 223  5061.40291 8.048069e+07          NaN
## 224  5084.20202 8.120737e+07          NaN
## 225  5107.00113 8.193732e+07          NaN
## 226  5129.80024 8.267054e+07          NaN
## 227  5152.59936 8.340702e+07          NaN
## 228  5175.39847 8.414677e+07          NaN
## 229  5198.19758 8.488979e+07          NaN
## 230  5220.99669 8.563607e+07          NaN
## 231  5243.79580 8.638561e+07          NaN
## 232  5266.59492 8.713842e+07          NaN
## 233  5289.39403 8.789450e+07          NaN
## 234  5312.19314 8.865385e+07          NaN
## 235  5334.99225 8.941646e+07          NaN
## 236  5357.79137 9.018233e+07          NaN
## 237  5380.59048 9.095148e+07          NaN
## 238  5403.38959 9.172388e+07          NaN
## 239  5426.18870 9.249956e+07          NaN
## 240  5448.98781 9.327850e+07          NaN
## 241  5471.78693 9.406070e+07          NaN
## 242  5494.58604 9.484618e+07          NaN
## 243  5517.38515 9.563491e+07          NaN
## 244  5540.18426 9.642692e+07          NaN
## 245  5562.98338 9.722219e+07          NaN
## 246  5585.78249 9.802073e+07          NaN
## 247  5608.58160 9.882253e+07          NaN
## 248  5631.38071 9.962760e+07          NaN
## 249  5654.17982 1.004359e+08          NaN
## 250  5676.97894 1.012475e+08          NaN
## 251  5699.77805 1.020624e+08          NaN
## 252  5722.57716 1.028805e+08          NaN
## 253  5745.37627 1.037019e+08          NaN
## 254  5768.17539 1.045266e+08          NaN
## 255  5790.97450 1.053545e+08          NaN
## 256  5813.77361 1.061857e+08          NaN
## 257  5836.57272 1.070202e+08          NaN
## 258  5859.37183 1.078579e+08          NaN
## 259  5882.17095 1.086989e+08          NaN
## 260  5904.97006 1.095432e+08          NaN
## 261  5927.76917 1.103907e+08          NaN
## 262  5950.56828 1.112415e+08          NaN
## 263  5973.36739 1.120955e+08          NaN
## 264  5996.16651 1.129529e+08          NaN
## 265  6018.96562 1.138135e+08          NaN
## 266  6041.76473 1.146773e+08          NaN
## 267  6064.56384 1.155444e+08          NaN
## 268  6087.36296 1.164148e+08          NaN
## 269  6110.16207 1.172885e+08          NaN
## 270  6132.96118 1.181654e+08          NaN
## 271  6155.76029 1.190456e+08          NaN
## 272  6178.55940 1.199290e+08          NaN
## 273  6201.35852 1.208157e+08          NaN
## 274  6224.15763 1.217057e+08          NaN
## 275  6246.95674 1.225990e+08          NaN
## 276  6269.75585 1.234955e+08          NaN
## 277  6292.55497 1.243953e+08          NaN
## 278  6315.35408 1.252983e+08          NaN
## 279  6338.15319 1.262046e+08          NaN
## 280  6360.95230 1.271142e+08          NaN
## 281  6383.75141 1.280271e+08          NaN
## 282  6406.55053 1.289432e+08          NaN
## 283  6429.34964 1.298626e+08          NaN
## 284  6452.14875 1.307852e+08          NaN
## 285  6474.94786 1.317111e+08          NaN
## 286  6497.74698 1.326403e+08          NaN
## 287  6520.54609 1.335727e+08          NaN
## 288  6543.34520 1.345084e+08          NaN
## 289  6566.14431 1.354474e+08          NaN
## 290  6588.94342 1.363897e+08          NaN
## 291  6611.74254 1.373352e+08          NaN
## 292  6634.54165 1.382839e+08          NaN
## 293  6657.34076 1.392360e+08          NaN
## 294  6680.13987 1.401913e+08          NaN
## 295  6702.93898 1.411498e+08          NaN
## 296  6725.73810 1.421117e+08          NaN
## 297  6748.53721 1.430768e+08          NaN
## 298  6771.33632 1.440452e+08          NaN
## 299  6794.13543 1.450168e+08          NaN
## 300  6816.93455 1.459917e+08          NaN
## 301  6839.73366 1.469699e+08          NaN
## 302  6862.53277 1.479513e+08          NaN
## 303  6885.33188 1.489360e+08          NaN
## 304  6908.13099 1.499239e+08          NaN
## 305  6930.93011 1.509152e+08          NaN
## 306  6953.72922 1.519097e+08          NaN
## 307  6976.52833 1.529074e+08          NaN
## 308  6999.32744 1.539085e+08          NaN
## 309  7022.12656 1.549128e+08          NaN
## 310  7044.92567 1.559203e+08          NaN
## 311  7067.72478 1.569311e+08          NaN
## 312  7090.52389 1.579452e+08          NaN
## 313  7113.32300 1.589626e+08          NaN
## 314  7136.12212 1.599832e+08          NaN
## 315  7158.92123 1.610071e+08          NaN
## 316  7181.72034 1.620343e+08          NaN
## 317  7204.51945 1.630647e+08          NaN
## 318  7227.31857 1.640984e+08          NaN
## 319  7250.11768 1.651353e+08          NaN
## 320  7272.91679 1.661755e+08          NaN
## 321  7295.71590 1.672190e+08          NaN
## 322  7318.51501 1.682658e+08          NaN
## 323  7341.31413 1.693158e+08          NaN
## 324  7364.11324 1.703691e+08          NaN
## 325  7386.91235 1.714256e+08          NaN
## 326  7409.71146 1.724855e+08          NaN
## 327  7432.51058 1.735485e+08          NaN
## 328  7455.30969 1.746149e+08          NaN
## 329  7478.10880 1.756845e+08          NaN
## 330  7500.90791 1.767574e+08          NaN
## 331  7523.70702 1.778335e+08          NaN
## 332  7546.50614 1.789129e+08          NaN
## 333  7569.30525 1.799956e+08          NaN
## 334  7592.10436 1.810816e+08          NaN
## 335  7614.90347 1.821708e+08          NaN
## 336  7637.70258 1.832632e+08          NaN
## 337  7660.50170 1.843590e+08          NaN
## 338  7683.30081 1.854580e+08          NaN
## 339  7706.09992 1.865603e+08          NaN
## 340  7728.89903 1.876658e+08          NaN
## 341  7751.69815 1.887746e+08          NaN
## 342  7774.49726 1.898867e+08          NaN
## 343  7797.29637 1.910020e+08          NaN
## 344  7820.09548 1.921206e+08          NaN
## 345  7842.89459 1.932425e+08          NaN
## 346  7865.69371 1.943676e+08          NaN
## 347  7888.49282 1.954960e+08          NaN
## 348  7911.29193 1.966277e+08          NaN
## 349  7934.09104 1.977626e+08          NaN
## 350  7956.89016 1.989008e+08          NaN
## 351  7979.68927 2.000423e+08          NaN
## 352  8002.48838 2.011870e+08          NaN
## 353  8025.28749 2.023350e+08          NaN
## 354  8048.08660 2.034863e+08          NaN
## 355  8070.88572 2.046408e+08          NaN
## 356  8093.68483 2.057986e+08          NaN
## 357  8116.48394 2.069597e+08          NaN
## 358  8139.28305 2.081240e+08          NaN
## 359  8162.08217 2.092916e+08          NaN
## 360  8184.88128 2.104625e+08          NaN
## 361  8207.68039 2.116366e+08          NaN
## 362  8230.47950 2.128140e+08          NaN
## 363  8253.27861 2.139946e+08          NaN
## 364  8276.07773 2.151786e+08          NaN
## 365  8298.87684 2.163657e+08          NaN
## 366  8321.67595 2.175562e+08          NaN
## 367  8344.47506 2.187499e+08          NaN
## 368  8367.27418 2.199469e+08          NaN
## 369  8390.07329 2.211472e+08          NaN
## 370  8412.87240 2.223507e+08          NaN
## 371  8435.67151 2.235575e+08          NaN
## 372  8458.47062 2.247675e+08          NaN
## 373  8481.26974 2.259808e+08          NaN
## 374  8504.06885 2.271974e+08          NaN
## 375  8526.86796 2.284173e+08          NaN
## 376  8549.66707 2.296404e+08          NaN
## 377  8572.46618 2.308668e+08          NaN
## 378  8595.26530 2.320964e+08          NaN
## 379  8618.06441 2.333293e+08          NaN
## 380  8640.86352 2.345655e+08          NaN
## 381  8663.66263 2.358050e+08          NaN
## 382  8686.46175 2.370477e+08          NaN
## 383  8709.26086 2.382937e+08          NaN
## 384  8732.05997 2.395429e+08          NaN
## 385  8754.85908 2.407954e+08          NaN
## 386  8777.65819 2.420512e+08          NaN
## 387  8800.45731 2.433102e+08          NaN
## 388  8823.25642 2.445725e+08          NaN
## 389  8846.05553 2.458381e+08          NaN
## 390  8868.85464 2.471069e+08          NaN
## 391  8891.65376 2.483790e+08          NaN
## 392  8914.45287 2.496544e+08          NaN
## 393  8937.25198 2.509331e+08          NaN
## 394  8960.05109 2.522150e+08          NaN
## 395  8982.85020 2.535001e+08          NaN
## 396  9005.64932 2.547886e+08          NaN
## 397  9028.44843 2.560803e+08          NaN
## 398  9051.24754 2.573752e+08          NaN
## 399  9074.04665 2.586735e+08          NaN
## 400  9096.84577 2.599750e+08          NaN
## 401  9119.64488 2.612797e+08          NaN
## 402  9142.44399 2.625878e+08          NaN
## 403  9165.24310 2.638991e+08          NaN
## 404  9188.04221 2.652136e+08          NaN
## 405  9210.84133 2.665315e+08          NaN
## 406  9233.64044 2.678526e+08          NaN
## 407  9256.43955 2.691769e+08          NaN
## 408  9279.23866 2.705045e+08          NaN
## 409  9302.03778 2.718354e+08          NaN
## 410  9324.83689 2.731696e+08          NaN
## 411  9347.63600 2.745070e+08          NaN
## 412  9370.43511 2.758477e+08          NaN
## 413  9393.23422 2.771917e+08          NaN
## 414  9416.03334 2.785389e+08          NaN
## 415  9438.83245 2.798894e+08          NaN
## 416  9461.63156 2.812431e+08          NaN
## 417  9484.43067 2.826002e+08          NaN
## 418  9507.22978 2.839604e+08          NaN
## 419  9530.02890 2.853240e+08          NaN
## 420  9552.82801 2.866908e+08          NaN
## 421  9575.62712 2.880609e+08          NaN
## 422  9598.42623 2.894343e+08          NaN
## 423  9621.22535 2.908109e+08          NaN
## 424  9644.02446 2.921908e+08          NaN
## 425  9666.82357 2.935739e+08          NaN
## 426  9689.62268 2.949603e+08          NaN
## 427  9712.42179 2.963500e+08          NaN
## 428  9735.22091 2.977430e+08          NaN
## 429  9758.02002 2.991392e+08          NaN
## 430  9780.81913 3.005386e+08          NaN
## 431  9803.61824 3.019414e+08          NaN
## 432  9826.41736 3.033474e+08          NaN
## 433  9849.21647 3.047567e+08          NaN
## 434  9872.01558 3.061692e+08          NaN
## 435  9894.81469 3.075850e+08          NaN
## 436  9917.61380 3.090041e+08          NaN
## 437  9940.41292 3.104265e+08          NaN
## 438  9963.21203 3.118521e+08          NaN
## 439  9986.01114 3.132809e+08          NaN
## 440 10008.81025 3.147131e+08          NaN
## 441 10031.60937 3.161485e+08          NaN
## 442 10054.40848 3.175872e+08          NaN
## 443 10077.20759 3.190291e+08          NaN
## 444 10100.00670 3.204743e+08          NaN
## 445 10122.80581 3.219228e+08          NaN
## 446 10145.60493 3.233745e+08          NaN
## 447 10168.40404 3.248295e+08          NaN
## 448 10191.20315 3.262878e+08          NaN
## 449 10214.00226 3.277493e+08          NaN
## 450 10236.80137 3.292141e+08          NaN
## 451 10259.60049 3.306822e+08          NaN
## 452 10282.39960 3.321535e+08          NaN
## 453 10305.19871 3.336281e+08          NaN
## 454 10327.99782 3.351060e+08          NaN
## 455 10350.79694 3.365871e+08          NaN
## 456 10373.59605 3.380715e+08          NaN
## 457 10396.39516 3.395591e+08          NaN
## 458 10419.19427 3.410501e+08          NaN
## 459 10441.99338 3.425443e+08          NaN
## 460 10464.79250 3.440417e+08          NaN
## 461 10487.59161 3.455424e+08          NaN
## 462 10510.39072 3.470464e+08          NaN
## 463 10533.18983 3.485537e+08          NaN
## 464 10555.98895 3.500642e+08          NaN
## 465 10578.78806 3.515780e+08          NaN
## 466 10601.58717 3.530951e+08          NaN
## 467 10624.38628 3.546154e+08          NaN
## 468 10647.18539 3.561390e+08          NaN
## 469 10669.98451 3.576658e+08          NaN
## 470 10692.78362 3.591959e+08          NaN
## 471 10715.58273 3.607293e+08          NaN
## 472 10738.38184 3.622660e+08          NaN
## 473 10761.18096 3.638059e+08          NaN
## 474 10783.98007 3.653491e+08          NaN
## 475 10806.77918 3.668955e+08          NaN
## 476 10829.57829 3.684453e+08          NaN
## 477 10852.37740 3.699982e+08          NaN
## 478 10875.17652 3.715545e+08          NaN
## 479 10897.97563 3.731140e+08          NaN
## 480 10920.77474 3.746768e+08          NaN
## 481 10943.57385 3.762428e+08          NaN
## 482 10966.37297 3.778121e+08          NaN
## 483 10989.17208 3.793847e+08          NaN
## 484 11011.97119 3.809606e+08          NaN
## 485 11034.77030 3.825397e+08          NaN
## 486 11057.56941 3.841220e+08          NaN
## 487 11080.36853 3.857077e+08          NaN
## 488 11103.16764 3.872966e+08          NaN
## 489 11125.96675 3.888888e+08          NaN
## 490 11148.76586 3.904842e+08          NaN
## 491 11171.56497 3.920829e+08          NaN
## 492 11194.36409 3.936849e+08          NaN
## 493 11217.16320 3.952901e+08          NaN
## 494 11239.96231 3.968986e+08          NaN
## 495 11262.76142 3.985104e+08          NaN
## 496 11285.56054 4.001254e+08          NaN
## 497 11308.35965 4.017437e+08          NaN
## 498 11331.15876 4.033653e+08          NaN
## 499 11353.95787 4.049901e+08          NaN
## 500 11376.75698 4.066182e+08          NaN
## 501 11399.55610 4.082496e+08          NaN
## 502 11422.35521 4.098842e+08          NaN
## 503 11445.15432 4.115221e+08          NaN
## 504 11467.95343 4.131633e+08          NaN
## 505 11490.75255 4.148077e+08          NaN
## 506 11513.55166 4.164554e+08          NaN
## 507 11536.35077 4.181064e+08          NaN
## 508 11559.14988 4.197606e+08          NaN
## 509 11581.94899 4.214181e+08          NaN
## 510 11604.74811 4.230788e+08          NaN
## 511 11627.54722 4.247429e+08          NaN
## 512 11650.34633 4.264102e+08          NaN
## 513 11673.14544 4.280807e+08          NaN

From the graph, we can see that for all distances, SQF locations appear to be clustered up until a distance of approximately 3000. Ripleys k shows us if we have spatial clusters present in our point data, but it does not tell us WHERE clusters are occurring. To discover this we can use DBSAN

DBSCAN

Results of the Ripley’s K analysis earlier shows we are getting clustering up to a radius of around 3000m, with the largest bulge in the graph also being at around 3000m radius . Therefore, 3000m is probably a good place to start and we will begin by searching for clusters of at least 4. if this distance isnt suitable, i will explore the elbow plot and use my own judgement to set eps.

Upon attempting the clusters at 3000, it seems to pick up all the points, therefore i have adjuster the eps distance to 900 as per the elbow plot. Moreover, having the cluster threshold at 4, you can see clusters emerge.

#first check the coordinate reference system of the this spatial polygon:
st_geometry(police)

## Geometry set for 77 features 
## Geometry type: MULTIPOLYGON
## Dimension:     XY
## Bounding box:  xmin: 563069.7 ymin: 4483098 xmax: 609762.3 ymax: 4529952
## Projected CRS: WGS 84 / UTM zone 18N
## First 5 geometries:

## MULTIPOLYGON (((580787.3 4504805, 580819.3 4504...

## MULTIPOLYGON (((585412.9 4508493, 585408.9 4508...

## MULTIPOLYGON (((584460.3 4510216, 584702 451008...

## MULTIPOLYGON (((590567.2 4502074, 590552.2 4502...

## MULTIPOLYGON (((584652.4 4502815, 584661.5 4502...

#1.extract the points from the 'spatial points (sp)' data frame
sqf_black_sp_extract <- sqf_black_sp%>% #takes 2A above
  coordinates(.)%>%
  as.data.frame()

#2. run the DBSCAN using the extracted sp points: 
#i choose clustering of eviction to be at 900 and min number of 4 points
DB_sqf_black <- sqf_black_sp_extract  %>% 
  fpc::dbscan(.,eps = 900, MinPts = 4) 

#now plot the results
plot(DB_sqf_black, sqf_black_sp_extract , main = "SQF-Black DBSCAN Output", frame = F)
plot(police$geometry, add=T)

#3. lets double check appropriate eps distance with the elbow plot:

sqf_black_sp_extract%>% #takes the point extracted from sp object (1. above)
  dbscan::kNNdistplot(.,k=4)

#shows 4000, howveer this seems too large so i'll keep it at 1000 as selected earlier

The outcome of the dbsan confirms that clustering of SQF on black people seems to follow a similarity in the density distribution mapped earlier. Using this map, we can not only confirm there there is clusters of SQF on black people happening, but that the highest clusters are occuring in the same areas mentioned earlier-manhattan, Brooklyn and Bronx.

Could these precincts be influenced by neighboring areas that could perhaps explain their clustering? let explore this spatial autocorrelation using morans I.

07 Spatial Autocorrelation: SQF Black people

According to Waldo R. Tobler, “Everything is related to everything else. But near things are more related than distant things”. Lets investigate precinct-level spatial dependency using localized version of morana’s I in order to see hot spots of where SQF programs on black people are happening.

If they exibit spatial autocorrelation then this would mean that precinct spatial units are influenced by other neighboring precincts and hence may explain potential clusters happening in areas that are geographically close.If this is the case, then similar policies for neighbouring precincts that can be adopted.

Spatial weight matrix

Before performing our clustered analysis, we need to generate a spatial weights matrix

#1. calculate the centroids of all polygons in NYC
centroids_police <- police_sqf_black%>%
  st_centroid()%>%
  st_geometry()

## Warning in st_centroid.sf(.): st_centroid assumes attributes are constant over
## geometries of x

plot(centroids_police)

#2. generate neighbor lists

#queen's case neighbors : doesn't take centroids. taken adjacent neighbors.
NL_queens <- police_sqf_black %>%   
  poly2nb(., queen=T)

#or k nearest neighbors : take centroids. 
NL_KNN <-centroids_police %>%  
  knearneigh(., k=4)

NL_KNN <- NL_KNN %>%
  knn2nb()

#now plot them both: Fist take the NL then the centroid of the spatial object
plot(NL_queens, st_geometry(centroids_police), col="red")
plot(police_sqf_black$geometry, add=T)

plot(NL_KNN, st_geometry(centroids_police), col="blue")
#add a map underneath
plot(police_sqf_black$geometry, add=T)

#3. create a spatial weights matrix object from these weights
# using row standardization 'w'

SWM_queens <- NL_queens %>%
  nb2listw(., style="W") 

SWM_knn <-NL_KNN %>%
  nb2listw(., style="W")

Local Moran’s I

final output map 1:

tmap_mode("plot")

## tmap mode set to plotting

#1. Local morans density
local_moran_density <-  police_sqf_black %>%
  pull(density_numeric) %>% #column name from police_sqf_black use the density values or count to get local morans
  as.vector()%>%
  localmoran(., SWM_knn)%>%  #spatial weight matrix of knn (generated from before from spatial data)
  as_tibble()
  

slice_head(local_moran_density, n=5) #view the values

## # A tibble: 5 x 5
##   Ii         E.Ii          Var.Ii      Z.Ii       `Pr(z != E(Ii))`
##   <localmrn> <localmrn>    <localmrn>  <localmrn> <localmrn>      
## 1 0.2268173  -0.0010295780 0.019007011 1.6526698  0.09839808      
## 2 0.8454488  -0.0154123401 0.280430301 1.6256260  0.10402924      
## 3 0.3645525  -0.0079308681 0.145400076 0.9768424  0.32864717      
## 4 0.1584960  -0.0005050172 0.009328005 1.6462868  0.09970473      
## 5 0.5370529  -0.0104965722 0.191940564 1.2497984  0.21137319

# 2. Copy the I score (column 1) and the z-score sd. (column 4)) back into police_sqf_black data

police_sqf_black <- police_sqf_black %>%
  mutate(density_I =as.numeric(local_moran_density$Ii))%>% #local_moran_density is tibble above, specific col =Ii
  mutate(density_Iz =as.numeric(local_moran_density$Z.Ii))


#3.lets map them out:

#set breaks
breaks1<-c(-1000,-2.58,-1.96,-1.65,1.65,1.96,2.58,1000)

#set colors
MoranColours<- rev(brewer.pal(8, "RdGy"))



tmap_mode("plot")

## tmap mode set to plotting

#plot interactive map
tm_shape(police_sqf_black) +
  tm_layout( title = "Local Moran's I- 
SQF:Black People")+
  tm_layout(frame = FALSE)+
  tm_polygons("density_Iz",
        style="fixed",
        breaks=breaks1,
        palette=MoranColours,
        midpoint=NA,
        title="Local Moran's I",
        legend.hist= TRUE)+
  tm_compass() +
  tm_scale_bar() +
  tm_legend(position=c(1,0.0))+
  tm_credits("(c) New York City Police Department", position=c(0.5,0.0))

The limitation of using local morans i alsone is that it is susceptible to shape and sizes of the spatial units therefore in order to overcome this issues, this map should be read along side the dbscan map that shows spatial clusters.

This local morans I map shows rare cases of the highest hot spots of SQF on black people occurring in precincts: 23,24,26,28 and 30. By reading this map along with the dbscan cluster analysis, we can see that these precincts not only exhibit high clusters of SQF cases on black people but also have high influence on each other.

08 White People SQF

• observe clustering of white people SQF and compare it to black people
• Overlay points of of SQF White people to spatial autocorrelation data maps of SQF black
• Develop local moran’s I map for SQF white people data and compare the spatial autocorrelation

# first filter only white and drop any na vals in lat and long. this will be used later to compare if i have time
sqf_white <-  sqf%>%
  filter(suspect_race_description == "WHITE HISPANIC" | suspect_race_description == "WHITE")%>%
  filter(month2 == "June" | month2 == "July" | month2 == "August" | month2 == "September" )%>%
  drop_na(stop_location_x)%>%
  drop_na(stop_location_y)

  

#2) transform csv sqf_white to point data: st_as sf. 
sqf_white_sf <- sqf_white%>%
  st_as_sf(.,coords = c ( "stop_location_x", "stop_location_y"), crs=2908)%>%
  st_transform(.,32618)

  
#make all points distinct 
sqf_white_sf <- sqf_white_sf%>%
  distinct(geometry, .keep_all=T)


#3) overlay point data to police districts and clip if required
tmap_mode("view")

## tmap mode set to interactive viewing

tm_shape(police) +
  tm_polygons(col = NA, alpha = 0.5) +
tm_shape(sqf_white_sf) +
  tm_dots(col = "blue")

#clipping point to xxx map boundary
sqf_white_sf <- sqf_white_sf[police,]

#plot again to see if they are clipped
tmap_mode("view")

## tmap mode set to interactive viewing

tm_shape(police) +
  tm_polygons(col = NA, alpha = 0.5) +
tm_shape(sqf_white_sf) +
  tm_dots(col = "blue")

Density map of white people

#1) observe areas of highest and lowest density of sqf-white


#st_join point data to the police map to allow for continuous data observation
police_sqf_white <- police%>%
  st_join(sqf_white_sf)

#count no. of points in each precinct and add area and density 
police_sqf_white <- police_sqf_white%>%
  add_count(precinct)%>% #counts the no. of points in each precinct (n)
  janitor::clean_names()%>%
  mutate(area = st_area(.))%>% #get area
  mutate (density = n/area)%>%  #get density
  dplyr::select(density, area, precinct, n) #narror down the colomns you want


#group by borough
police_sqf_white <-  police_sqf_white%>%
  group_by(precinct)%>%
  summarise(density = first(density), precinct = first(precinct), area = first(area), 
            count = first(n))



#quick choropleth map based on DENSITY  
tm_shape(police_sqf_white) +
    tm_polygons("density",
        style="jenks",
        palette="PuOr",
        midpoint=NA,
        popup.vars=c("precinct", "density"),
        title="SQF-White people Density")

we can see some slight differences when compared to density of black people, however they both show high density in Manhattan. Lets explore the clusters of SQF white people in order to make further comparisons.

Cluster analysis: White people

ANalysis process: 1. create sp and sf objects 2. Density kernel (which confirms the density map created above) 3. Ripley K (over quadrat) analysis to test clustering present 4. DBSCAN to show where clusters are happening

1.Create sp and ppp objects from the sf point data in order to proceed with ripley k and DBSCAN

#1) now set a window as the precinct nyc boundary 
window <- as.owin(police)
plot(window)

#2A)  Create a sp object for eviction (sf to sp)
sqf_white_sp <- sqf_white_sf  %>%
  as(., 'Spatial')

#2B) Create a ppp object
sqf_white_ppp <- ppp(x=sqf_white_sp@coords[,1],
                          y=sqf_white_sp@coords[,2],
                          window=window)

#sqf_black_sp@coords[,1]



#3) have a look at the ppp object

sqf_white_ppp %>%
  plot(.,pch=16,cex=0.5, 
       main="SQF white people")

kernel Density:

This confirms the density map produced earlier.

#uses ppp data 
sqf_white_ppp %>%
  density(., sigma=500) %>%
  plot()

We can see from this map that the similar to SQF black people cases, the precincts that fall in manhattan and part of Bronx also shows highest densities of SQF concentration for white people

Ripleys k

#using the ppp data, plot the ripleys K
Ripleys_K <- sqf_white_ppp%>%
  Kest(., correction="border") %>%
  plot()

Kval_global <- as.data.frame(Kest(sqf_black_ppp, correction = "border"))
Kval_global

##               r         theo       border
## 1       0.00000 0.000000e+00         0.00
## 2      22.79911 1.632998e+03     11901.04
## 3      45.59822 6.531993e+03     54863.40
## 4      68.39734 1.469699e+04     93369.07
## 5      91.19645 2.612797e+04    156446.40
## 6     113.99556 4.082496e+04    228775.26
## 7     136.79467 5.878794e+04    328605.76
## 8     159.59379 8.001692e+04    447167.33
## 9     182.39290 1.045119e+05    560608.39
## 10    205.19201 1.322729e+05    666790.28
## 11    227.99112 1.632998e+05    822915.30
## 12    250.79023 1.975928e+05    967560.83
## 13    273.58935 2.351518e+05   1130273.23
## 14    296.38846 2.759767e+05   1336499.26
## 15    319.18757 3.200677e+05   1509229.26
## 16    341.98668 3.674246e+05   1683968.37
## 17    364.78580 4.180476e+05   1891542.38
## 18    387.58491 4.719365e+05   2116443.25
## 19    410.38402 5.290915e+05   2371491.31
## 20    433.18313 5.895124e+05   2632645.29
## 21    455.98224 6.531993e+05   2862262.06
## 22    478.78136 7.201523e+05   3107248.72
## 23    501.58047 7.903712e+05   3310106.09
## 24    524.37958 8.638561e+05   3625339.91
## 25    547.17869 9.406070e+05   3884404.02
## 26    569.97780 1.020624e+06   4267395.80
## 27    592.77692 1.103907e+06   4551358.65
## 28    615.57603 1.190456e+06   4921733.80
## 29    638.37514 1.280271e+06   5231634.26
## 30    661.17425 1.373352e+06   5552652.60
## 31    683.97337 1.469699e+06   5895016.32
## 32    706.77248 1.569311e+06   6227699.18
## 33    729.57159 1.672190e+06   6614419.70
## 34    752.37070 1.778335e+06   6977882.03
## 35    775.16981 1.887746e+06   7264778.61
## 36    797.96893 2.000423e+06   7575702.49
## 37    820.76804 2.116366e+06   7935309.18
## 38    843.56715 2.235575e+06   8316194.52
## 39    866.36626 2.358050e+06   8677560.44
## 40    889.16538 2.483790e+06   9095402.71
## 41    911.96449 2.612797e+06   9511941.25
## 42    934.76360 2.745070e+06   9997693.18
## 43    957.56271 2.880609e+06  10363077.15
## 44    980.36182 3.019414e+06  10861423.31
## 45   1003.16094 3.161485e+06  11372886.99
## 46   1025.96005 3.306822e+06  11704053.34
## 47   1048.75916 3.455424e+06  12302945.07
## 48   1071.55827 3.607293e+06  12874746.77
## 49   1094.35739 3.762428e+06  13291070.80
## 50   1117.15650 3.920829e+06  13750427.22
## 51   1139.95561 4.082496e+06  14236622.60
## 52   1162.75472 4.247429e+06  14649409.33
## 53   1185.55383 4.415628e+06  15070560.45
## 54   1208.35295 4.587092e+06  15574522.07
## 55   1231.15206 4.761823e+06  16114500.91
## 56   1253.95117 4.939820e+06  16491338.94
## 57   1276.75028 5.121083e+06  16884198.59
## 58   1299.54940 5.305612e+06  17346373.34
## 59   1322.34851 5.493406e+06  18054670.59
## 60   1345.14762 5.684467e+06  18682228.85
## 61   1367.94673 5.878794e+06  19417468.72
## 62   1390.74584 6.076387e+06  19991285.51
## 63   1413.54496 6.277246e+06  20505256.11
## 64   1436.34407 6.481370e+06  20858307.53
## 65   1459.14318 6.688761e+06  21690633.95
## 66   1481.94229 6.899418e+06  22543926.56
## 67   1504.74140 7.113341e+06  22932505.59
## 68   1527.54052 7.330530e+06  23937425.05
## 69   1550.33963 7.550984e+06  24784454.00
## 70   1573.13874 7.774705e+06  25466470.51
## 71   1595.93785 8.001692e+06  26453468.20
## 72   1618.73697 8.231945e+06  26809224.38
## 73   1641.53608 8.465463e+06  27512515.29
## 74   1664.33519 8.702248e+06  28624783.48
## 75   1687.13430 8.942299e+06  29119030.14
## 76   1709.93341 9.185616e+06  30126251.23
## 77   1732.73253 9.432198e+06  31056845.79
## 78   1755.53164 9.682047e+06  31790604.24
## 79   1778.33075 9.935162e+06  32569564.17
## 80   1801.12986 1.019154e+07  33403977.91
## 81   1823.92898 1.045119e+07  33828068.89
## 82   1846.72809 1.071410e+07  34607635.48
## 83   1869.52720 1.098028e+07  36294671.33
## 84   1892.32631 1.124973e+07  37191279.34
## 85   1915.12542 1.152244e+07  37939674.77
## 86   1937.92454 1.179841e+07  39174179.11
## 87   1960.72365 1.207766e+07  39849630.24
## 88   1983.52276 1.236016e+07  41004097.35
## 89   2006.32187 1.264594e+07  42151433.69
## 90   2029.12099 1.293498e+07  42820802.52
## 91   2051.92010 1.322729e+07  43693489.00
## 92   2074.71921 1.352286e+07  45434396.65
## 93   2097.51832 1.382170e+07  46057820.81
## 94   2120.31743 1.412380e+07  47466476.98
## 95   2143.11655 1.442917e+07  49506160.26
## 96   2165.91566 1.473781e+07  50369084.22
## 97   2188.71477 1.504971e+07  51424931.00
## 98   2211.51388 1.536488e+07  54062187.64
## 99   2234.31299 1.568332e+07  55736129.46
## 100  2257.11211 1.600502e+07  58072904.50
## 101  2279.91122 1.632998e+07  60650153.12
## 102  2302.71033 1.665822e+07  62315730.92
## 103  2325.50944 1.698971e+07  63759731.22
## 104  2348.30856 1.732448e+07  64776632.83
## 105  2371.10767 1.766251e+07  67522267.19
## 106  2393.90678 1.800381e+07  68868785.19
## 107  2416.70589 1.834837e+07  69661526.88
## 108  2439.50500 1.869620e+07  70494748.83
## 109  2462.30412 1.904729e+07  72233911.34
## 110  2485.10323 1.940165e+07  73081329.35
## 111  2507.90234 1.975928e+07  74403301.45
## 112  2530.70145 2.012017e+07  75521893.22
## 113  2553.50057 2.048433e+07  76329519.43
## 114  2576.29968 2.085176e+07  77708968.58
## 115  2599.09879 2.122245e+07  78805453.80
## 116  2621.89790 2.159640e+07  79577173.60
## 117  2644.69701 2.197363e+07  80009819.01
## 118  2667.49613 2.235411e+07  81352129.14
## 119  2690.29524 2.273787e+07  81713694.16
## 120  2713.09435 2.312489e+07  81860579.95
## 121  2735.89346 2.351518e+07  82049433.11
## 122  2758.69258 2.390873e+07  84355900.07
## 123  2781.49169 2.430555e+07  85670049.85
## 124  2804.29080 2.470563e+07  86796463.94
## 125  2827.08991 2.510898e+07  86233256.89
## 126  2849.88902 2.551560e+07  85907848.38
## 127  2872.68814 2.592548e+07  87697595.22
## 128  2895.48725 2.633863e+07  88493038.26
## 129  2918.28636 2.675504e+07  90300863.35
## 130  2941.08547 2.717473e+07  90300863.35
## 131  2963.88459 2.759767e+07  91811688.61
## 132  2986.68370 2.802388e+07 104944246.60
## 133  3009.48281 2.845336e+07 106300115.42
## 134  3032.28192 2.888611e+07 108062744.88
## 135  3055.08103 2.932212e+07 109337261.57
## 136  3077.88015 2.976139e+07 108740679.29
## 137  3100.67926 3.020394e+07 110638895.64
## 138  3123.47837 3.064975e+07          NaN
## 139  3146.27748 3.109882e+07          NaN
## 140  3169.07659 3.155116e+07          NaN
## 141  3191.87571 3.200677e+07          NaN
## 142  3214.67482 3.246564e+07          NaN
## 143  3237.47393 3.292778e+07          NaN
## 144  3260.27304 3.339318e+07          NaN
## 145  3283.07216 3.386185e+07          NaN
## 146  3305.87127 3.433379e+07          NaN
## 147  3328.67038 3.480899e+07          NaN
## 148  3351.46949 3.528746e+07          NaN
## 149  3374.26860 3.576920e+07          NaN
## 150  3397.06772 3.625420e+07          NaN
## 151  3419.86683 3.674246e+07          NaN
## 152  3442.66594 3.723400e+07          NaN
## 153  3465.46505 3.772879e+07          NaN
## 154  3488.26417 3.822686e+07          NaN
## 155  3511.06328 3.872819e+07          NaN
## 156  3533.86239 3.923279e+07          NaN
## 157  3556.66150 3.974065e+07          NaN
## 158  3579.46061 4.025178e+07          NaN
## 159  3602.25973 4.076617e+07          NaN
## 160  3625.05884 4.128383e+07          NaN
## 161  3647.85795 4.180476e+07          NaN
## 162  3670.65706 4.232895e+07          NaN
## 163  3693.45618 4.285641e+07          NaN
## 164  3716.25529 4.338713e+07          NaN
## 165  3739.05440 4.392112e+07          NaN
## 166  3761.85351 4.445838e+07          NaN
## 167  3784.65262 4.499890e+07          NaN
## 168  3807.45174 4.554269e+07          NaN
## 169  3830.25085 4.608975e+07          NaN
## 170  3853.04996 4.664007e+07          NaN
## 171  3875.84907 4.719365e+07          NaN
## 172  3898.64819 4.775050e+07          NaN
## 173  3921.44730 4.831062e+07          NaN
## 174  3944.24641 4.887401e+07          NaN
## 175  3967.04552 4.944066e+07          NaN
## 176  3989.84463 5.001057e+07          NaN
## 177  4012.64375 5.058376e+07          NaN
## 178  4035.44286 5.116021e+07          NaN
## 179  4058.24197 5.173992e+07          NaN
## 180  4081.04108 5.232290e+07          NaN
## 181  4103.84019 5.290915e+07          NaN
## 182  4126.63931 5.349866e+07          NaN
## 183  4149.43842 5.409144e+07          NaN
## 184  4172.23753 5.468748e+07          NaN
## 185  4195.03664 5.528679e+07          NaN
## 186  4217.83576 5.588937e+07          NaN
## 187  4240.63487 5.649521e+07          NaN
## 188  4263.43398 5.710432e+07          NaN
## 189  4286.23309 5.771669e+07          NaN
## 190  4309.03220 5.833233e+07          NaN
## 191  4331.83132 5.895124e+07          NaN
## 192  4354.63043 5.957341e+07          NaN
## 193  4377.42954 6.019885e+07          NaN
## 194  4400.22865 6.082756e+07          NaN
## 195  4423.02777 6.145953e+07          NaN
## 196  4445.82688 6.209476e+07          NaN
## 197  4468.62599 6.273326e+07          NaN
## 198  4491.42510 6.337503e+07          NaN
## 199  4514.22421 6.402007e+07          NaN
## 200  4537.02333 6.466837e+07          NaN
## 201  4559.82244 6.531993e+07          NaN
## 202  4582.62155 6.597477e+07          NaN
## 203  4605.42066 6.663286e+07          NaN
## 204  4628.21978 6.729423e+07          NaN
## 205  4651.01889 6.795886e+07          NaN
## 206  4673.81800 6.862676e+07          NaN
## 207  4696.61711 6.929792e+07          NaN
## 208  4719.41622 6.997235e+07          NaN
## 209  4742.21534 7.065004e+07          NaN
## 210  4765.01445 7.133100e+07          NaN
## 211  4787.81356 7.201523e+07          NaN
## 212  4810.61267 7.270272e+07          NaN
## 213  4833.41179 7.339348e+07          NaN
## 214  4856.21090 7.408750e+07          NaN
## 215  4879.01001 7.478479e+07          NaN
## 216  4901.80912 7.548535e+07          NaN
## 217  4924.60823 7.618917e+07          NaN
## 218  4947.40735 7.689626e+07          NaN
## 219  4970.20646 7.760661e+07          NaN
## 220  4993.00557 7.832023e+07          NaN
## 221  5015.80468 7.903712e+07          NaN
## 222  5038.60379 7.975727e+07          NaN
## 223  5061.40291 8.048069e+07          NaN
## 224  5084.20202 8.120737e+07          NaN
## 225  5107.00113 8.193732e+07          NaN
## 226  5129.80024 8.267054e+07          NaN
## 227  5152.59936 8.340702e+07          NaN
## 228  5175.39847 8.414677e+07          NaN
## 229  5198.19758 8.488979e+07          NaN
## 230  5220.99669 8.563607e+07          NaN
## 231  5243.79580 8.638561e+07          NaN
## 232  5266.59492 8.713842e+07          NaN
## 233  5289.39403 8.789450e+07          NaN
## 234  5312.19314 8.865385e+07          NaN
## 235  5334.99225 8.941646e+07          NaN
## 236  5357.79137 9.018233e+07          NaN
## 237  5380.59048 9.095148e+07          NaN
## 238  5403.38959 9.172388e+07          NaN
## 239  5426.18870 9.249956e+07          NaN
## 240  5448.98781 9.327850e+07          NaN
## 241  5471.78693 9.406070e+07          NaN
## 242  5494.58604 9.484618e+07          NaN
## 243  5517.38515 9.563491e+07          NaN
## 244  5540.18426 9.642692e+07          NaN
## 245  5562.98338 9.722219e+07          NaN
## 246  5585.78249 9.802073e+07          NaN
## 247  5608.58160 9.882253e+07          NaN
## 248  5631.38071 9.962760e+07          NaN
## 249  5654.17982 1.004359e+08          NaN
## 250  5676.97894 1.012475e+08          NaN
## 251  5699.77805 1.020624e+08          NaN
## 252  5722.57716 1.028805e+08          NaN
## 253  5745.37627 1.037019e+08          NaN
## 254  5768.17539 1.045266e+08          NaN
## 255  5790.97450 1.053545e+08          NaN
## 256  5813.77361 1.061857e+08          NaN
## 257  5836.57272 1.070202e+08          NaN
## 258  5859.37183 1.078579e+08          NaN
## 259  5882.17095 1.086989e+08          NaN
## 260  5904.97006 1.095432e+08          NaN
## 261  5927.76917 1.103907e+08          NaN
## 262  5950.56828 1.112415e+08          NaN
## 263  5973.36739 1.120955e+08          NaN
## 264  5996.16651 1.129529e+08          NaN
## 265  6018.96562 1.138135e+08          NaN
## 266  6041.76473 1.146773e+08          NaN
## 267  6064.56384 1.155444e+08          NaN
## 268  6087.36296 1.164148e+08          NaN
## 269  6110.16207 1.172885e+08          NaN
## 270  6132.96118 1.181654e+08          NaN
## 271  6155.76029 1.190456e+08          NaN
## 272  6178.55940 1.199290e+08          NaN
## 273  6201.35852 1.208157e+08          NaN
## 274  6224.15763 1.217057e+08          NaN
## 275  6246.95674 1.225990e+08          NaN
## 276  6269.75585 1.234955e+08          NaN
## 277  6292.55497 1.243953e+08          NaN
## 278  6315.35408 1.252983e+08          NaN
## 279  6338.15319 1.262046e+08          NaN
## 280  6360.95230 1.271142e+08          NaN
## 281  6383.75141 1.280271e+08          NaN
## 282  6406.55053 1.289432e+08          NaN
## 283  6429.34964 1.298626e+08          NaN
## 284  6452.14875 1.307852e+08          NaN
## 285  6474.94786 1.317111e+08          NaN
## 286  6497.74698 1.326403e+08          NaN
## 287  6520.54609 1.335727e+08          NaN
## 288  6543.34520 1.345084e+08          NaN
## 289  6566.14431 1.354474e+08          NaN
## 290  6588.94342 1.363897e+08          NaN
## 291  6611.74254 1.373352e+08          NaN
## 292  6634.54165 1.382839e+08          NaN
## 293  6657.34076 1.392360e+08          NaN
## 294  6680.13987 1.401913e+08          NaN
## 295  6702.93898 1.411498e+08          NaN
## 296  6725.73810 1.421117e+08          NaN
## 297  6748.53721 1.430768e+08          NaN
## 298  6771.33632 1.440452e+08          NaN
## 299  6794.13543 1.450168e+08          NaN
## 300  6816.93455 1.459917e+08          NaN
## 301  6839.73366 1.469699e+08          NaN
## 302  6862.53277 1.479513e+08          NaN
## 303  6885.33188 1.489360e+08          NaN
## 304  6908.13099 1.499239e+08          NaN
## 305  6930.93011 1.509152e+08          NaN
## 306  6953.72922 1.519097e+08          NaN
## 307  6976.52833 1.529074e+08          NaN
## 308  6999.32744 1.539085e+08          NaN
## 309  7022.12656 1.549128e+08          NaN
## 310  7044.92567 1.559203e+08          NaN
## 311  7067.72478 1.569311e+08          NaN
## 312  7090.52389 1.579452e+08          NaN
## 313  7113.32300 1.589626e+08          NaN
## 314  7136.12212 1.599832e+08          NaN
## 315  7158.92123 1.610071e+08          NaN
## 316  7181.72034 1.620343e+08          NaN
## 317  7204.51945 1.630647e+08          NaN
## 318  7227.31857 1.640984e+08          NaN
## 319  7250.11768 1.651353e+08          NaN
## 320  7272.91679 1.661755e+08          NaN
## 321  7295.71590 1.672190e+08          NaN
## 322  7318.51501 1.682658e+08          NaN
## 323  7341.31413 1.693158e+08          NaN
## 324  7364.11324 1.703691e+08          NaN
## 325  7386.91235 1.714256e+08          NaN
## 326  7409.71146 1.724855e+08          NaN
## 327  7432.51058 1.735485e+08          NaN
## 328  7455.30969 1.746149e+08          NaN
## 329  7478.10880 1.756845e+08          NaN
## 330  7500.90791 1.767574e+08          NaN
## 331  7523.70702 1.778335e+08          NaN
## 332  7546.50614 1.789129e+08          NaN
## 333  7569.30525 1.799956e+08          NaN
## 334  7592.10436 1.810816e+08          NaN
## 335  7614.90347 1.821708e+08          NaN
## 336  7637.70258 1.832632e+08          NaN
## 337  7660.50170 1.843590e+08          NaN
## 338  7683.30081 1.854580e+08          NaN
## 339  7706.09992 1.865603e+08          NaN
## 340  7728.89903 1.876658e+08          NaN
## 341  7751.69815 1.887746e+08          NaN
## 342  7774.49726 1.898867e+08          NaN
## 343  7797.29637 1.910020e+08          NaN
## 344  7820.09548 1.921206e+08          NaN
## 345  7842.89459 1.932425e+08          NaN
## 346  7865.69371 1.943676e+08          NaN
## 347  7888.49282 1.954960e+08          NaN
## 348  7911.29193 1.966277e+08          NaN
## 349  7934.09104 1.977626e+08          NaN
## 350  7956.89016 1.989008e+08          NaN
## 351  7979.68927 2.000423e+08          NaN
## 352  8002.48838 2.011870e+08          NaN
## 353  8025.28749 2.023350e+08          NaN
## 354  8048.08660 2.034863e+08          NaN
## 355  8070.88572 2.046408e+08          NaN
## 356  8093.68483 2.057986e+08          NaN
## 357  8116.48394 2.069597e+08          NaN
## 358  8139.28305 2.081240e+08          NaN
## 359  8162.08217 2.092916e+08          NaN
## 360  8184.88128 2.104625e+08          NaN
## 361  8207.68039 2.116366e+08          NaN
## 362  8230.47950 2.128140e+08          NaN
## 363  8253.27861 2.139946e+08          NaN
## 364  8276.07773 2.151786e+08          NaN
## 365  8298.87684 2.163657e+08          NaN
## 366  8321.67595 2.175562e+08          NaN
## 367  8344.47506 2.187499e+08          NaN
## 368  8367.27418 2.199469e+08          NaN
## 369  8390.07329 2.211472e+08          NaN
## 370  8412.87240 2.223507e+08          NaN
## 371  8435.67151 2.235575e+08          NaN
## 372  8458.47062 2.247675e+08          NaN
## 373  8481.26974 2.259808e+08          NaN
## 374  8504.06885 2.271974e+08          NaN
## 375  8526.86796 2.284173e+08          NaN
## 376  8549.66707 2.296404e+08          NaN
## 377  8572.46618 2.308668e+08          NaN
## 378  8595.26530 2.320964e+08          NaN
## 379  8618.06441 2.333293e+08          NaN
## 380  8640.86352 2.345655e+08          NaN
## 381  8663.66263 2.358050e+08          NaN
## 382  8686.46175 2.370477e+08          NaN
## 383  8709.26086 2.382937e+08          NaN
## 384  8732.05997 2.395429e+08          NaN
## 385  8754.85908 2.407954e+08          NaN
## 386  8777.65819 2.420512e+08          NaN
## 387  8800.45731 2.433102e+08          NaN
## 388  8823.25642 2.445725e+08          NaN
## 389  8846.05553 2.458381e+08          NaN
## 390  8868.85464 2.471069e+08          NaN
## 391  8891.65376 2.483790e+08          NaN
## 392  8914.45287 2.496544e+08          NaN
## 393  8937.25198 2.509331e+08          NaN
## 394  8960.05109 2.522150e+08          NaN
## 395  8982.85020 2.535001e+08          NaN
## 396  9005.64932 2.547886e+08          NaN
## 397  9028.44843 2.560803e+08          NaN
## 398  9051.24754 2.573752e+08          NaN
## 399  9074.04665 2.586735e+08          NaN
## 400  9096.84577 2.599750e+08          NaN
## 401  9119.64488 2.612797e+08          NaN
## 402  9142.44399 2.625878e+08          NaN
## 403  9165.24310 2.638991e+08          NaN
## 404  9188.04221 2.652136e+08          NaN
## 405  9210.84133 2.665315e+08          NaN
## 406  9233.64044 2.678526e+08          NaN
## 407  9256.43955 2.691769e+08          NaN
## 408  9279.23866 2.705045e+08          NaN
## 409  9302.03778 2.718354e+08          NaN
## 410  9324.83689 2.731696e+08          NaN
## 411  9347.63600 2.745070e+08          NaN
## 412  9370.43511 2.758477e+08          NaN
## 413  9393.23422 2.771917e+08          NaN
## 414  9416.03334 2.785389e+08          NaN
## 415  9438.83245 2.798894e+08          NaN
## 416  9461.63156 2.812431e+08          NaN
## 417  9484.43067 2.826002e+08          NaN
## 418  9507.22978 2.839604e+08          NaN
## 419  9530.02890 2.853240e+08          NaN
## 420  9552.82801 2.866908e+08          NaN
## 421  9575.62712 2.880609e+08          NaN
## 422  9598.42623 2.894343e+08          NaN
## 423  9621.22535 2.908109e+08          NaN
## 424  9644.02446 2.921908e+08          NaN
## 425  9666.82357 2.935739e+08          NaN
## 426  9689.62268 2.949603e+08          NaN
## 427  9712.42179 2.963500e+08          NaN
## 428  9735.22091 2.977430e+08          NaN
## 429  9758.02002 2.991392e+08          NaN
## 430  9780.81913 3.005386e+08          NaN
## 431  9803.61824 3.019414e+08          NaN
## 432  9826.41736 3.033474e+08          NaN
## 433  9849.21647 3.047567e+08          NaN
## 434  9872.01558 3.061692e+08          NaN
## 435  9894.81469 3.075850e+08          NaN
## 436  9917.61380 3.090041e+08          NaN
## 437  9940.41292 3.104265e+08          NaN
## 438  9963.21203 3.118521e+08          NaN
## 439  9986.01114 3.132809e+08          NaN
## 440 10008.81025 3.147131e+08          NaN
## 441 10031.60937 3.161485e+08          NaN
## 442 10054.40848 3.175872e+08          NaN
## 443 10077.20759 3.190291e+08          NaN
## 444 10100.00670 3.204743e+08          NaN
## 445 10122.80581 3.219228e+08          NaN
## 446 10145.60493 3.233745e+08          NaN
## 447 10168.40404 3.248295e+08          NaN
## 448 10191.20315 3.262878e+08          NaN
## 449 10214.00226 3.277493e+08          NaN
## 450 10236.80137 3.292141e+08          NaN
## 451 10259.60049 3.306822e+08          NaN
## 452 10282.39960 3.321535e+08          NaN
## 453 10305.19871 3.336281e+08          NaN
## 454 10327.99782 3.351060e+08          NaN
## 455 10350.79694 3.365871e+08          NaN
## 456 10373.59605 3.380715e+08          NaN
## 457 10396.39516 3.395591e+08          NaN
## 458 10419.19427 3.410501e+08          NaN
## 459 10441.99338 3.425443e+08          NaN
## 460 10464.79250 3.440417e+08          NaN
## 461 10487.59161 3.455424e+08          NaN
## 462 10510.39072 3.470464e+08          NaN
## 463 10533.18983 3.485537e+08          NaN
## 464 10555.98895 3.500642e+08          NaN
## 465 10578.78806 3.515780e+08          NaN
## 466 10601.58717 3.530951e+08          NaN
## 467 10624.38628 3.546154e+08          NaN
## 468 10647.18539 3.561390e+08          NaN
## 469 10669.98451 3.576658e+08          NaN
## 470 10692.78362 3.591959e+08          NaN
## 471 10715.58273 3.607293e+08          NaN
## 472 10738.38184 3.622660e+08          NaN
## 473 10761.18096 3.638059e+08          NaN
## 474 10783.98007 3.653491e+08          NaN
## 475 10806.77918 3.668955e+08          NaN
## 476 10829.57829 3.684453e+08          NaN
## 477 10852.37740 3.699982e+08          NaN
## 478 10875.17652 3.715545e+08          NaN
## 479 10897.97563 3.731140e+08          NaN
## 480 10920.77474 3.746768e+08          NaN
## 481 10943.57385 3.762428e+08          NaN
## 482 10966.37297 3.778121e+08          NaN
## 483 10989.17208 3.793847e+08          NaN
## 484 11011.97119 3.809606e+08          NaN
## 485 11034.77030 3.825397e+08          NaN
## 486 11057.56941 3.841220e+08          NaN
## 487 11080.36853 3.857077e+08          NaN
## 488 11103.16764 3.872966e+08          NaN
## 489 11125.96675 3.888888e+08          NaN
## 490 11148.76586 3.904842e+08          NaN
## 491 11171.56497 3.920829e+08          NaN
## 492 11194.36409 3.936849e+08          NaN
## 493 11217.16320 3.952901e+08          NaN
## 494 11239.96231 3.968986e+08          NaN
## 495 11262.76142 3.985104e+08          NaN
## 496 11285.56054 4.001254e+08          NaN
## 497 11308.35965 4.017437e+08          NaN
## 498 11331.15876 4.033653e+08          NaN
## 499 11353.95787 4.049901e+08          NaN
## 500 11376.75698 4.066182e+08          NaN
## 501 11399.55610 4.082496e+08          NaN
## 502 11422.35521 4.098842e+08          NaN
## 503 11445.15432 4.115221e+08          NaN
## 504 11467.95343 4.131633e+08          NaN
## 505 11490.75255 4.148077e+08          NaN
## 506 11513.55166 4.164554e+08          NaN
## 507 11536.35077 4.181064e+08          NaN
## 508 11559.14988 4.197606e+08          NaN
## 509 11581.94899 4.214181e+08          NaN
## 510 11604.74811 4.230788e+08          NaN
## 511 11627.54722 4.247429e+08          NaN
## 512 11650.34633 4.264102e+08          NaN
## 513 11673.14544 4.280807e+08          NaN

From the graph, we can see that up to a distance of 2100, SQF locations for white people appear to be clustered and from about 2100 to 2400 they are dispersed.. Lets perform dbsan

DBSCAN

i’ll begin with eps of 2100 and adjust if needed.

Upon attempting the clusters at 2100, it seems to pick up all the points, therefore i have adjuster the eps distance to 900 (even though elbow plot shows 500) i order to match the same distance as black people. This will allows a fair comparison between the two.

#first check the coordinate reference system of the this spatial polygon:
st_geometry(police)

## Geometry set for 77 features 
## Geometry type: MULTIPOLYGON
## Dimension:     XY
## Bounding box:  xmin: 563069.7 ymin: 4483098 xmax: 609762.3 ymax: 4529952
## Projected CRS: WGS 84 / UTM zone 18N
## First 5 geometries:

## MULTIPOLYGON (((580787.3 4504805, 580819.3 4504...

## MULTIPOLYGON (((585412.9 4508493, 585408.9 4508...

## MULTIPOLYGON (((584460.3 4510216, 584702 451008...

## MULTIPOLYGON (((590567.2 4502074, 590552.2 4502...

## MULTIPOLYGON (((584652.4 4502815, 584661.5 4502...

#1.extract the points from the 'spatial points (sp)' data frame
sqf_white_sp_extract <- sqf_white_sp%>% 
  coordinates(.)%>%
  as.data.frame()

#2. run the DBSCAN using the extracted sp points: 
#i choose clustering of eviction to be at 100 and min number of 6 points
DB_sqf_white <- sqf_white_sp_extract  %>% 
  fpc::dbscan(.,eps = 900, MinPts = 4) 

#now plot the results
plot(DB_sqf_white, sqf_white_sp_extract , main = "SQF-White DBSCAN Output", frame = F)
plot(police$geometry, add=T)

#3. lets double check appropriate eps distance with the elbow plot:

sqf_white_sp_extract%>% #takes the point extracted from sp object (1. above)
  dbscan::kNNdistplot(.,k=4)

#shows 4000, howveer this seems too large so i'll keep it at 1000 as selected earlier

The outcome of the dbsan confirms that clustering of SQF on white people seems

fOR the same eps distances, SQF clusters of black people are more apparent in the precincts that fall inside Brooklyn, However there is a similarity in clustering of both black and white SQF cases in Manhattan.

Lets explore the spatial autocorrelation of sqf white people to make a comparison.

Local Morans

Spatial weight matrix: Before performing our clustered analysis, we need to generate a spatial weights matrix

#1. calculate the centroids of all polygons in NYC
centroids_police_w <- police_sqf_white%>%
  st_centroid()%>%
  st_geometry()

## Warning in st_centroid.sf(.): st_centroid assumes attributes are constant over
## geometries of x

plot(centroids_police_w)

#2. generate neighbor lists

#queen's case neighbors : doesn't take centroids. taken adjacent neighbors.
NL_queens_W <- police_sqf_white %>%   
  poly2nb(., queen=T)

#or k nearest neighbors : take centroids. 
NL_KNN_w <-centroids_police_w %>%  
  knearneigh(., k=4)

NL_KNN_w <- NL_KNN_w %>%
  knn2nb()

#now plot them both: Fist take the NL then the centroid of the spatial object
plot(NL_queens_W, st_geometry(centroids_police_w), col="red")
plot(police_sqf_white$geometry, add=T)

plot(NL_KNN_w, st_geometry(centroids_police_w), col="blue")
#add a map underneath
plot(police_sqf_white$geometry, add=T)

#3. create a spatial weights matrix object from these weights
# using row standardization 'w'

SWM_queens_w <- NL_queens_W %>%
  nb2listw(., style="W") 


SWM_knn <-NL_KNN %>%
  nb2listw(., style="W")

Local Moran’s I

final output map 2:

tmap_mode("plot")

## tmap mode set to plotting

#first make sure density col is numeric
police_sqf_white$density_numeric <- as.numeric(police_sqf_white$density)



#1. Local morans density
local_moran_density_w <-  police_sqf_white %>%
  pull(density_numeric) %>% #column name from police_sqf_black use the density values or count to get local morans
  as.vector()%>%
  localmoran(., SWM_knn)%>%  #spatial weight matrix of knn (generated from before from spatial data)
  as_tibble()
  

slice_head(local_moran_density_w, n=5) #view the values

## # A tibble: 5 x 5
##   Ii         E.Ii         Var.Ii     Z.Ii       `Pr(z != E(Ii))`
##   <localmrn> <localmrn>   <localmrn> <localmrn> <localmrn>      
## 1 0.4498512  -0.001430091 0.02639028 2.7779558  0.005470206     
## 2 1.0910988  -0.013745052 0.25051721 2.2074054  0.027285748     
## 3 0.7334375  -0.011358632 0.20752325 1.6349483  0.102059891     
## 4 0.7253888  -0.008842327 0.16196131 1.8244297  0.068087143     
## 5 0.3616623  -0.007785350 0.14275316 0.9778224  0.328162159

# 2. Copy the I score (column 1) and the z-score sd. (column 4)) back into police_sqf_black data

police_sqf_white <- police_sqf_white %>%
  mutate(density_I =as.numeric(local_moran_density_w$Ii))%>% #local_moran_density is tibble above, specific col =Ii
  mutate(density_Iz =as.numeric(local_moran_density_w$Z.Ii))


#3.lets map them out:

#set breaks
breaks1<-c(-1000,-2.58,-1.96,-1.65,1.65,1.96,2.58,1000)

#set colors
MoranColours<- rev(brewer.pal(8, "RdGy"))



tmap_mode("plot")

## tmap mode set to plotting

#plot interactive map
tm_shape(police_sqf_white) +
  tm_layout( title = "Local Moran's I- 
SQF:white People")+
  tm_layout(frame = FALSE)+
  tm_polygons("density_Iz",
        style="fixed",
        breaks=breaks1,
        palette=MoranColours,
        midpoint=NA,
        title="Local Moran's I",
        legend.hist= TRUE)+
  tm_compass() +
  tm_scale_bar() +
  tm_legend(position=c(1,0.0))+
  tm_credits("(c) New York City Police Department", position=c(0.5,0.0))

final output map 3:

#set breaks
breaks1<-c(-1000,-2.58,-1.96,-1.65,1.65,1.96,2.58,1000)

#set colors
MoranColours<- rev(brewer.pal(8, "RdGy"))


#map SQF: black people
tmap_mode("plot")

## tmap mode set to plotting

#plot interactive map
tm1 <-  tm_shape(police_sqf_black) +
  tm_layout( title = "Local Moran's: 
Black People")+
  tm_polygons("density_Iz",
        style="fixed",
        breaks=breaks1,
        palette=MoranColours,
        midpoint=NA,
        title="Local Moran's I SQF:Black People")+
  tm_legend(show=FALSE)+
  tm_layout(frame=FALSE)



#map SQF: white people
tmap_mode("plot")

## tmap mode set to plotting

#plot interactive map
tm2 <- tm_shape(police_sqf_white) +
  tm_layout( title = "Local Moran's:
White People")+
  tm_polygons("density_Iz",
        style="fixed",
        breaks=breaks1,
        palette=MoranColours,
        midpoint=NA,
        title="Local Moran's I")+
  tm_legend(show=FALSE)+
  tm_layout(frame=FALSE)



legend1 <- tm_shape(police_sqf_black) +
    tm_polygons("density_Iz",
        style="fixed",
        breaks=breaks1,
        palette=MoranColours,
        midpoint=NA,
        title="SQF:Black people",
        legend.hist= FALSE) +
    tm_scale_bar(position=c(0.2,0.04), text.size=0.6)+
    tm_compass(north=0, position=c(0,0.1))+
    tm_layout(legend.only = TRUE, legend.position=c(0.5,0.25),asp=0.1)+
    tm_credits("(c) New York City Police Department", position=c(0.0,0.0))



#map the legend
legend <- tm_shape(police_sqf_white) +
    tm_polygons("density_Iz",
        style="fixed",
        breaks=breaks1,
        palette=MoranColours,
        midpoint=NA,
        title="SQF:White people",
        legend.hist= FALSE) +
    tm_layout(legend.only = TRUE, legend.position=c(0.5,0.25),asp=0.1)
    




#plot them side by side
t=tmap_arrange(tm1, tm2, legend1, legend, ncol=2)


t

Comparing the 2 maps we can see that some precincts exhibit similarities in levels of autocorrelation specifically in staten island (1.6-2) and manhattan. In general we can see high levels of spatial autocorrelation in both maps in precincts inside manhattan which could indicate that although theyr are both high, there isnt necessarily racial-profiling happening in manhattan.

09 Reflection

The analysis conducted lead us to reach a conclusion that answers the research question and both hypothesis

Ripleys k indicated spatial clustering was present In both cases of black people and white people.

DBSAN was used to observe clustered pattern and local morans to determine hot spots of spatial autocorrelation. The analysis was first done for all SQF on suspect black people and results showed that clustering was most present in precincts across Manhattan, west of Bronx and north of Brooklyn. The clustering performed on SQF of white cases showed that for a similar eps value, there were similarities in clustering in Manhattan, however sqf clustering of black people were higher in Brooklyn which indicates there may be possibilities of racial profiling of black people in Brooklyn

This Lead us to confirm H1 for the first hypothesis:There are noticeable spacial cluster pattern of SQF policy that can be observed for the black people across NYC

Moreover, by cross comparing Morans i of both black and white sqf cases showed some precincts exhibit similarities in spatial autocorrelation, with highest levels of spatial autocorrelation for both maps in precincts inside manhattan. This could indicate there there may be other factors such internal governance within these precincts that are causing such strong spatial similarities between these precincts.

This confirms H1 in hypothesis 2: H1: There are are similarities in spatial autocorrelation of (SQF) policy between black and white people

Limitations to the analysis - Ripley’s k is sensitive to the boundaries of the study area - Addition spatial autocorrelation was not explored including gerrys and getis -The eps distance outcome of ripleys K could not be used as it was too large and so 900m assumption was set for both morans i cases. -NYPD collected the data so there may have been a confilict of interest

Policy Making -in areas where there are clusters of SQR on black people such as the bronx, policies to ensure that there is no police violence can be be implemented. This may include internal trainin of police forces within these precincts to ensure that racial profiling and racial profiling does not occur.

Further Exploration -clusters over different periods from 2018 compared to 2020 to compare changes after black lives matter. -clusters of socio economic factors that may influence or be linked to SQF racial cases -local regression models for each cluster:localized analysis and therefore custom policy making for precinct that exhibit similarities in spatial pattern